CLUSTERED SPARSITY AND 
SEPARATION OF CARTOON AND TEXTURE 



GITTA KUTYNIOK 

Abstract. Natural images are typically a composition of cartoon and texture structures. 
A medical image might, for instance, show a mixture of gray matter and the skull cap. One 
common task is to separate such an image into two single images, one containing the cartoon 
part and the other containing the texture part. Recently, a powerful class of algorithms using 
sparse approximation and £± minimization has been introduced to resolve this problem, and 
numerous inspiring empirical results have already been obtained. 

In this paper we provide the first thorough theoretical study of the separation of a combi- 
nation of cartoon and texture structures in a model situation using this class of algorithms. 
The methodology we consider expands the image in a combined dictionary consisting of a 
curvelet tight frame and a Gabor tight frame and minimizes the l\ norm on the analysis 
side. Sparse approximation properties then force the cartoon components into the curvelet 
coefficients and the texture components into the Gabor coefficients, thereby separating the 
image. Utilizing the fact that the coefficients are clustered geometrically, we prove that at 
sufficiently fine scales arbitrarily precise separation is possible. Main ingredients of our anal- 
ysis are the novel notion of cluster coherence and clustered/geometric sparsity. Our analysis 
also provides a deep understanding on when separation is still possible. 



1. Introduction 

Natural images are typically a composition of 'cartoon' and 'texture'. Think, for instance, 
of a medical image in which we might see a mixture of gray matter and the skull cap, or, 
more general, a mixture of tissue and bones. Often, it is essential to very cleanly separate 
the cartoon from the texture part, i.e., to generate two single images from the original one, 
for separate analysis. However, each educated person these days would say that it is entirely 
impossible to solve a problem with only one known datum and two unknowns. Intriguingly, 
the deep reason for why this separation is possible in the considered situation is the stark 
morphological difference between both structures. 

The main idea of the empirical results exploiting applied harmonic analysis methodologies 
(see, e.g., [49, 50, 51, 52, 25] - in comparison to PDE-based separation methods (see exem- 
plary [56] and references therein), use the fact that there exist sparsifying dictionaries for 
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cartoons and for textures. Dictionary learning on natural images indicates that a curvelet 
(or shearlet) system might be best adapted to the cartoon part, whereas a Gabor system 
might be best adapted to the texture part (cf. [46]). Roughly speaking, the image is then 
expanded into a combined dictionary of curvelets and a Gabor system, and the l\ norm 
on the analysis side is minimized. Sparse approximation properties then force the cartoon 
components into the curvelet coefficients and the texture components into the Gabor coef- 
ficients, thereby separating the image. However, no theoretical results are available to date 
which provide a deep mathematical understanding of why separation is possible. 

In the paper [19] (see also [38]), the apparently similar problem of separating point- 
and curve-like structures was considered, a distributional model was developed, and clus- 
tered/geometric sparsity and cluster coherence were introduced to derive an asymptotic 
separation result; asymptotic in the scale. However, the analysis required for the situation 
of cartoon and texture differs significantly, since a Gabor system does not exhibit a scaling 
component, and a distributional model is not feasible for the texture part. Our analysis 
will though still be based on the novel viewpoint of clustered/geometric sparsity and cluster 
coherence. 

There exists an intriguing connection with quasicrystals [28], since those objects can be re- 
garded as both periodic (texture-like) as well as non-periodic with sharp boundaries (cartoon- 
like). This raises the question of how periodization of a cartoon can make it separable from 
a single cartoon. This highly fascinating question is though beyond the scope of this paper, 
but one goal for future exploration. 

1.1. Model of Cartoon Part. Intuitively, cartoons are smooth image parts separated from 
other areas by an edge. The first model of cartoons has been introduced in [5], and this is 
what we intend to use also here. The basic idea is to choose a closed boundary curve and 
then fill the interior and exterior part with C 2 functions. Here, we will adapt the definition 
from [5] slightly. 

We define a model C of a cartoon as follows: Let r be a closed, non-intersecting, and 
regular curve in R 2 , i.e., r is C°° and of finite arclength, and let B T denote the interior of r. 
Then we choose C as 

C = /„ + /i-1b t , 

where /o,/i £ L 2 (M?) n C 2 (R 2 ) with compact support. An example of such a cartoon is 
illustrated in Figure 1. 




Figure 1. Example of a cartoon-like image. 
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To set up the analysis in a reasonable way, we will need to restrict our model of cartoons 
slightly, which will be done in Subsection 1.5. 

1.2. Model of Texture Part. People have debated for years over an appropriate model for 
the texture content of an image. The outcome can be seen nowadays in an extensive variety 
of texture models both for the deterministic as well as statistical setting. We would like to 
remind the reader of models such as Gaussian random fields, or the i/ 1 -model [47]. All of 
those emphasize different features of texture content such as for the if _1 -model a special 
oscillatory behavior 'dual' to cartoons, which are sometimes modeled as -B^-functions, is 
highlighted. However, certainly even an intersection of all these models is far from being 
capable of accurately describing texture, especially since the perception of texture also differs 
from observer to observer. In the area of computer vision very interesting studies have been 
performed, for instance by Zucker [2, 22, 23], which we would like to bring to the reader's 
attention. 

Our understanding of texture is a function which is concentrated on a small part of the 
image with a periodic structure. This justifies the use of a very 'classical' approach by 
modeling texture as a function which is sparse in a Gabor frame. However, we would like the 
reader to be aware of the fact that certainly also this deterministic model is quite simplified 
and possible extensions could be incorporated in it, which we would like to briefly discuss: 

• Warping: Let us consider an image containing a table cloth such as the famous 
'Barbara' image. Remembering that this table cloth was in fact folded, we could 
think of improving our model by using diffeomorphism to model this warping effect, 
which was done in [10]. 

• Affine transform: Again abusing 'Barbara' as an example, we recall that the table 
cloth was in addition also sheared. This could be handled by application of an affine 
transform to the texture content. 

• Windowing: Our running example also reminds us that texture - here a pattern of a 
table cloth - could be contained in a polygon, e.g., in particular, the pattern is broken. 
Thus refining our model further would force us to consider windowed 'Gabor' texture 
models, which, in particular, requires a careful treatment of the boundaries. This 
also illustrates how delicate the distinction of texture from cartoons can be, and let 
us wonder where in such a transition to draw the line between texture and cartoons. 

• Textons: Finally, textures commonly do not appear as singletons, but might also 
overlap. This can be attacked by considering a model of texture which is a linear 
combination of shifted 'Gabor' texture model with coefficients being iid to decouple 
the texture parts. As a reference for this general concept which in literature is 
normally referred to as textons, we would like to mention [57] . 

We now define our model for texture as follows: Let g G L 2 (M. 2 ) be a window with 
\g(x) \ ~ e - ' 1 '!, g G C 00 ^ 2 ), and frequency support supp g C [—1, l] 2 satisfying 

For a 'size parameter' s > 0, let g s denote the L 2 -normalized scaled version of g defined by 

g s (x) = s ■ g(sx). 
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With this choice, g s (0 = s 1 ■ g(s and hence 

supp g s C [-s, s} 2 

as well as 

]T|^(£ + sn)| 2 = s - 2 , £gM 2 . 

Further, let (<im,n)m,nez 2 be a sequence of complex numbers belonging to t\ D £ 2 - Then the 
function of interest to us as a model for texture is defined by 

m,n£Z 2 

1.3. Model of Image Composed of a Cartoon- and Texture-Part. Having introduced 
both constituents, we now assume that the image 

f s = C + % (1) 

is observed, however both the cartoon component C as well as the texture component T s are 
unknown to us. Thus we face the task of extracting those from f s , which is what we will 
analyze in this paper. 

For the purpose of matching the energy of cartoon- and texture-part so that the separation 
becomes non-trivial, in Subsection 1.5, we will link s to a particular other parameter. 

1.4. Sparsifying Overcomplete Systems. Since we intend to utilize i\ minimization to 
separate C and T s in the sense of extracting C and T s from f s , appropriate systems need to be 
selected which provide (relatively) sparse representations for those components. Of interest 
to us are the following two systems: 

• Curvelets - a highly directional tight frame with increasingly anisotropic elements at 
fine scales. 

• Gabor tight frame - a tight frame with time-frequency-balanced elements. 

These systems are constructed as follows. Using the function g introduced in Subsection 
1.2, we first define the Gabor tight frame at spatial position index m and frequency position 
index n by the Fourier transform 

(g s )x(0 =# s (£-sn)e^S 

where we let A = (m, n) index spatial and frequency position. The bandsize is indexed by 
the parameter s. It follows from [12] that this system constitutes a tight frame for L 2 (IR 2 ). 

To define curvelets, we let W be the inverse Fourier transform of a wavelet, where W be- 
longs to C°°(R) and is compactly supported on [1/2, 2], for instance, suitably scaled Lemarie- 
Meyer wavelets possess these properties. Further, we choose a "bump function" V G C°°, 
which is compactly supported on [—1, 1]. We then define continuous curvelets at scale a > 0, 
orientation 9 G [0, it), and spatial position b G M 2 by their Fourier transforms 

%,„A0 = J ■ W{a\i\)V{a- l ' 2 {uj - 9)) ■ e^. 
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See [5, 6] for more details. The curvelet tight frame is then (essentially) defined as a sampling 
of b on a series of regular lattices 

{R dje D a] Z 2 }, j>j , £ = 0,...,2^2J _i, 

where Re is planar rotation by 9 radians, aj = 2~\ 9jj = tt£/2^ 2 , £ = 0, . . . , 2^ 2 — 1, and D a 
is anisotropic dilation by diag(a, \/a), i.e., the curvelets at scale j, orientation £, and spatial 
position k = (k\, k2) are given by the Fourier transform 

= 2-ii ■ W{\t\/V)V{{u] - 6 3/ )2^ 2 ) ■ e^D^ky^ 

where rj = (j, k, £) index scale, orientation, and scale. (For a precise statement, see [7, Section 
4.3, pp. 210-211]). 

Using the same window W, we can construct a family of filters Fj with transfer functions 

These filters allow us to decompose a function / into pieces fj with different scales, the piece 
fj at subband j arises from filtering / using Ff. 

the Fourier transform fj is supported in the annulus with inner radius 2 J_1 and outer radius 
2 J+1 . Because of our assumption on W, we can reconstruct the original function from these 
pieces using the formula 

/ ^ /} * I,- feL 2 (R 2 ). 

j 

This allows us to split the seeked decomposition of the observed image f s (see (1)) into 
infinitely many decompositions depending on scale j by setting 

Cj = Fj*C and T sJ = Fj -k %, 

and, for each j, considering 

/>, ^ • T,.r (2) 
For each j, we assume that f s j is known to us, and we aim to compute Cj and T s j from it. 
By the choice of the filter, we can then derive C and T s from those. 

1.5. Frequency Matching. In order to set up the separation problem in a reasonable way, 
the two filtered components Cj and T s j have to be comparable as we go to finer scales, so 
that the ratio of energies is more or less independent of scale. This ensures that separation is 
challenging at every scale. Thus we need to compute the norms ||Cj||2 and ||7^j||2, and then 
link s to 2 J in such a way that both quantities are comparable. This will then also allow us 
to drive an asymptotic analysis based on the scale j - upon which the size s then depends. 

Computing the asymptotic behavior of the norm of our filtered cartoon model, we face 
the problem that the behavior of C is not always the same asymptotically for all possible 
elements. To resolve this problem, we restrict the model slightly to cartoons which satisfy 

ic(oi~ier 3/2 , eei 2 . (3) 

We wish to note that this is only a very mild restriction, since for the boundary of the sphere 
C = S 2 , it can be proven that this condition is satisfied (see, e.g., [6]). In general, Randol [48] 
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(cf. also [35]) proved, for instance, that if C G C 4 and if the Gaussian curvature is non-zero 
in every point of dC, then |C(f)| < C ■ |£|~ 3/2 , £ E M 2 . 

With this additional assumption, we can now derive the following lemma whose proof is 
outsourced to Subsection 4.1. 



Lemma 1.1 

'"j II 2 



C,-||3~2-', j^oo. 



We next compute the asymptotic behavior of the norm of our filtered texture model and 
also refer to Subsection 4.1 for the proof. 

Lemma 1.2. 

1 1 2 ^ — | to— m | r — ■> 

|| '7^,j || 2 ~ 2_^ e ~ dm,ndrh,n, 

where 

A sJ - 2 : — < i }. 

The energy matching condition on s and j can now be derived from Lemmata 1.1 and 1.2 
by choosing s = Sj to satisfy 

E|m-m| ^ — 
6 2 / ^m,n^m,n 2 ^ . (4) 

m,m n€Z 2 n.4 s . j 

3 ,J 

We now rewrite (2) as 

fi = C J + Tr (5) 
and also simply write Aj instead of A Sj j. 

For illustrative purposes we present the energy matching condition for one particular 
sequence (4,n)m,ngz 2 - 

Example 1.1. If we exemplarily choose 

\d m ,n\ ~ \m\- {2+S) ■ M" (2+5) , 5>0, 

and observe that for the asymptotics we can ignore the 'side terms' m ^ fa, the energy 
matching condition (4) becomes 

2~i ~J2\ m \~ (4+2S) ■ E \M~ (4+28) - (6) 

m n€Z 2 nAj 

Now |m|~( 4+2<5 ) is constant, and since 



2- 



2£±i /2jA -2(1+«) 



r -(4+25) . rdrd ^ 

2(1+5) 



o V^i 

3 



from (6), we finally obtain the condition 2 J ~ (2 J /sj) , hence we can choose Sj as 

1+25 • , , 

S . =2 2+2S3. (7) 

Thus, in particular, we have Sj = cu(2^ 2 ) and Sj = o(2 J ) in this case. This implies that 
energy matching is achieved, if the size of a repeated patch of the texture, i.e., 1/sj, is 
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'slightly' larger than the thickness of the filtered boundary of the cartoon, more precisely, 
between 2~i and 2~^ 2 . Hence intuitively separation seems doubtful in this case. It will thus 
be surprising that in Corollary 1.1, we will show arbitrarily precise separation with the scale 
j — > oo, if only 5 > 1. 

1.6. Separation via i\ Minimization. In our analysis we aim at exploring the geometrical 
contents and, in particular, their difference of the cartoonlike structure C and texturelike 
structure T s . In [19], we have introduced a fundamental notion which allows to derive 
estimates on the accuracy of i\ based separation of morphological objects. It was therein 
applied to prove that, given an image composed of pointlike and curvelike structures at all 
sufficiently fine scales, nearly-perfect separation can be achieved. Here we are concerned with 
a different situation, since such an asymptotical result is not possible, simply due to the fact 
that Gabor systems don't have a scaling parameter. Although our analysis will therefore be 
different in nature, the common bracket with [19] is again the utilization of the geometry of 
the components. This allows us to apply some of the novel techniques developed in [19] also 
in the setting considered in this paper. For the convenience of the reader, we will briefly 
recall those in this subsection. 

Suppose we have two tight frames $i, $2 in a Hilbert space T-L, and a signal vector 
S = Si + S® G H- We then consider the following optimization problem 

(Sep) (S?, S*) = argmm SljS J$f SiHi + \\Q$S 2 \\i subject to S = Si + S 2 . (8) 

We wish to remark that intentionally the norm is placed on the analysis coefficients rather 
than on the synthesis coefficients as in Basis Pursuit [9] to avoid self-terms in the frame 
setting. Hence we might not recover the most sparsest expansion, but it will turn out that 
this expansion is nevertheless sufficient for component separation. Also it is algorithmically 
not feasible to optimize over all possible expansions. In many studies of £± optimization, one 
then considers the mutual coherence 

\P) = max max \{<f>i, 

j i 

whose importance was shown by [18]. This might be thought of as singleton coherence. In 
contrast, to exploit the knowledge of the geometrical structure of the components - more 
precisely, the location of the 'large' coefficients of their frame expansions -, in [19] the notion 
of cluster coherence was introduced, which bounds coherence between a single member of a 
frame \I/ and a cluster of members of a second frame $, clustered at S. 

Definition 1.1. Given tight frames <3> = (<fri)i and $ = and an index subset S associ- 

ated with expansions in frame $ ; we define the cluster coherence 

(jl c (S,$]V) = max V . 

ies 

With this notion at hand, the following estimate concerning the accuracy of the extraction 
of S° and S® from the signal S holds true: 

Proposition 1.1. [19, Props. 1 + 2] Suppose that S can be decomposed as S = S® + S® so 
that each component Sf is relatively sparse in $j, i = 1,2, i.e., 

||l 5 c^ 1 || 1 +||l <S c$ 2 r S 2 °|| 1 <5. 
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Let (£*, £%) solve (8). Then 

9A 

IIC* oO 1 1 i || c* oO 1 1 ^ 

IPl — D l l|2 + 1 1 "2 ~~ D 2 112 — 1 — 2/i ' 

where 

H c = max(/x c (5i, $ i; $ 2 ), /i c (S 2 , $ 2 ; $i)). 

The concepts of this section will now be applied to (Sep), at scale j only. For this, the 
tight frames are $i, the full curvelet tight frame, and $2, the full Gabor tight frame, and S 
is our filtered observed image fj, which satisfies (cf. (5)), 

fj = Q + Tj. 

We apply the optimization problem (Sep), getting subsignal components S* and S%, which 
we then relabel as the curvelet component Cj and Gabor component Tj. 

The cluster of indices of both, sparse approximation error and cluster coherence, depends 
on the choice of the cluster indices S\ - now denoted by «Sij - for the curvelet system and the 
cluster indices <S 2 - now denoted by S 2 j - for the Gabor system. The sparse approximation 
error 5 and the cluster coherence /i c - let us remind the reader that these are merely analysis 
tools and are not part of the i\ minimization procedure - relevant for a scale j will then be 
denoted by 

5ij and 82 ,j 

for relative sparsity for the cartoon and texture part, respectively, as well as 

(ftc)ij = Hc(Sij, H); {{9s)\}) and (/i c ) 2 ,j = /i c (5 2J , {(# s )a}; {l v }) 
for the cluster coherence for a cluster of curvelets and Gabor elements, respectively. 

1.7. Asymptotic Separation Result. Our aim is an asymptotic separation result for the 
relative L 2 -error of the purported cartoon part Cj and texture part Tj decays to zero as 
j — > 00. By Proposition 1.1, this follows if the clusters S\j and S2J can be chosen such that 

max(5ij, <5 2 ,j) = o(2~^ 2 ), as j — > 00, 

and 

max((/i c )ij, (/i c ) 2 j) -> 0, as j ->■ 00. 

Our main result shows that this is indeed possible. For this, the set of significant coefficients 
S2J for Tj is defined by 

S 2 ,j =B(0,r ld ) x 5(0, r 2j ), 
for some rij,r 2 j > 0, where B(0,r) shall denote the closed £2 ball around the origin in M. 2 . 

Theorem 1.1. Assuming energy matching (4), we set 

A- = {^M 2 :^<|£|<^} 

and 

M o ., , s . = Re ■ sj^supp 7^,0,0 + #i(0, 1)) H Z 2 . 
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Suppose that there exist r lj -,r 2j ' > which satisfy 

E x — ^ |m — m\ . in 

\dm,n\e — = o(2~ J/i ), j oo 

and 

(i) in case Sj = o(2 J ) as j — > oo, 

\B(0, r 2j ) n M ajt0>Sj n Z 2 | = o(2 3 ^ 4 . sj 1 ), as j -»• oo, 

(ii) as we// as m case s^ = S7(2- 7 ) as j — > oo, 

(5(0, nj) n Z 2 | = o(2" 3i/4 ■ Sj -), as j oo. 
Tnen we have asymptotically near-perfect separation: 

iic*ii2+ra a 

Most interesting is the consideration of the special case already focussed on in Example 
1.1. In this situation asymptotically near-perfect separation is always attained provided that 
5 > 1. The precise statement is the following 

Corollary 1.1. Suppose that the sequence (c? m ,n)m,nez 2 satisfies 

iawi~H- (2+5) >r (2+5) , *>1- 

Then we have asymptotically near-perfect separation: 

K<7 — 2 + \\Tj — Tj 2 n 

J J ^-p- — > 0, j ->■ oo. 

||Cj|| 2 + ||7j|| 2 

Theorem 1.1 and Corollary 1.1 are both proved in Subsection 4.1. 

1.8. Interpretation. To heuristically understand why cartoon and texture can be separated 
at all, consider the worst case scenario that an image is composed of one cartoon and a 
periodized cartoon as is illustrated in Figure 2. 

Qi Qi Qi Qi Qi Qi Qi 
Q3Q3Q3Q3Q3Q3Q3 

Q3Q3 

03 03 
Q3Q3 

Qi Qi Qi Qi Qi Qi Qi 

Figure 2. Periodic small cartoons versus one large cartoon. 

The periodic cartoon is then of the form 

f=J2 1 c(x + k), CcR 2 . 

fcez 2 
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Taking the Fourier transform, we observe that 

/ = l c . e 2m{k '° 
fcez 2 

with the second part being responsible for the sparsity in a Gabor system. This sparsity is 
the key for separation from a single 'large' cartoon. A further question for this particular 
situation might be: Given a set C and a lattice AT?, how sparse is the corresponding 
periodization? However, this goes far beyond our model, wherefore we do not treat this 
question here, but label it an interesting direction for future research. 

Let us now take a closer look at Corollary 1.1. To first build up intuition on the energy 
balancing condition, we consider a curvelet of scale j and a Gabor element of size s. If s = Sj 
satisfies the energy balancing condition (7), the spatial footprint of the Gabor element is 
about the size of the curvelet, since Sj = oj (2^/2) and Sj = o{2i) in this case, which is 
illustrated in Figure 3. 




Figure 3. The essential spatial support of a curvelet ^j,k,e and Gabor element 
{9s)m,n satisfying the energy balancing condition s = Sj with Sj defined in (7). 

This also visualizes the relation between the filtered cartoon part Cj and the filtered texture 
part Tj{— Tsj.j), since those will in fact be similarly connected. It is intuitively clear that 
this is the 'worst scenario'. It seems at first sight quite astonishing that separation is still 
possible in this case. However, the geometrical clustering of the significant coefficients solves 
the mystery, and the notion of cluster coherence makes this precise. In fact, our result shows 
that the whole cluster of Gabor elements (g Sj )m,n with (m, n) e B(0, rij) x B(0, r2j) cannot 
be 'glued together' so that it generates a single curvelet. The same is true for the whole 
cluster of significant curvelets, which are unable to generate a single Gabor element. To 
visualize this would require to show the footprints of curvelets and Gabor elements in a 5 
dimensional 'phase space': Spatial domain M 2 , orientation [0,27r], and frequency R 2 . 

1.9. Extensions. Theorem 1.1 is amenable to the following generalizations and extensions. 

• Thresholding. Thresholding is a separation strategy, which is commonly utilized as 
a substitute for l\ minimization due to the fact that its algorithmic complexity is 
much lower. Hence one might ask: Can we derive a similar asymptotic separation 
result when using thresholding? And in fact, this is true. We though decided not 
to include the whole analysis, since it would go beyond the scope of this paper. We 
would just like to mention that similar techniques as in [39] are employable, but the 
technical details, which will now in addition require the special treatment of the scale 
of curvelets and the window size of Gabor atoms, will be very tedious. 
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• Intersections. Natural images are typically composed not only of one smoothly filled 
C 2 edge curve, but several with presumably various intersections. In a similar way as 
it is commented upon in [5], we can also argue here that intersections will not affect 
our analysis. Thus the framework can be extended to separating several cartoon-like 
objects from texture. 

• Other Systems. Theorem 1.1 holds without change for many other pairs of frames 
and bases, such as, e.g., by [40], for the pair or shearlets (cf. [31, 42, 37, 41]) and 
Gabor systems. 

• Noise. Theorem 1.1 is resilient to noise impact; an image composed of C and T with 
additive 'sufficiently small' noise exhibits the same asymptotic separation. This can 
be easily deduced by utilizing ideas from [19, Sect. 8]. 



2.1. Cartoon. Let Cj be the filtered version of the cartoon-part C of the considered image, 
where - as detailed in Subsection 1.4 - we use Fj(£) = W(a,j\£\) as a filter. We now describe 
how to partition Cj into smaller pieces which will then in a second step be bent to 'line 
objects' separately. The first part is inspired by a technique employed in [5], whereas the 
second part follows ideas introduced in [19]. 

We start by first smoothly localizing Cj near dyadic squares with a prescribed radius, 
which is chosen such that the curvature of each piece is controllable, but independent on j. 
More precisely, we define a partition of unity (wq)q £ q with those properties, and let 



We would like the reader to notice that the number of pieces C^q is finite and independent 
on j. Also there is no need to consider those C^q which are smooth, since their associated 
curvelet coefficients have sufficient decay. For each of the remaining functions Cj,q, without 
loss of generality we can assume - as was done in [5, Sect. 6.1] - that the edge discontinuity 
is centered at and that its first derivative equals zero pointing in the vertical direction. 
Hence, WLOG, the splitted cartoon might take a form as illustrated on the LHS of Figure 
4. 

Having partitioned the filtered cartoon Cj, we now apply a diffeomorphism 0q to each 
piece Cj t Q, which equals the identity outside of a compact set, thereby straightening out the 
discontinuity. For a similar strategy, we would like to refer the reader to [19, Sect. 7]. The 
resulting piece can now be modeled as 



where H = l Xl >o is the heaviside function and vq is a C°°-function supported in a dyadic 
square of sidelength p, say. An illustration of this slitting and bending is presented in Figure 
4. 

We then require the following lemma, which is stated slightly more general than needed. 
For this, we let H v be defined by 



2. Relative Sparsity 



C hQ - C i ■ W Q- 



H 3,Q = {H*Fj)-v Q , 
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image(r 




Figure 4. The filtered cartoon Cj and the mapping <f)q : Cj s Q — > Hj Q. 
hence, in particular, H 1 = H. 

Lemma 2.1. For each N = 1,2, ... , there is a constant cn so that 

\{{H V *F 3 ) ■ v Q , laAe )\ < c N ■ a- 1 '* • 1, log2 (o/o . )| <3 • ( 1 M ^ - M )~ N ■ (\h/a\)- N . 

Proof. First, we observe that 

\((H"*F j ) -VQ,-f aA e)\ < c a l|i og2 ( a/a3 )| <3 . (9) 

Therefore, we now restrict to the case a = dj. Next, using Parseval, and letting a>(£) denote 
the angular component of £, 

((H v *F j ).v Q , 7a , b , ) 
= (H u F j ,v Q *^ aAe ) 

= c- j "leil-" J "t)o(r)o 3 /V(a|6-r|)V'((a;(e 1 -r)-fl)/v^e- i <^- T >dT 

■W(a\Ci\)d^. (10) 

By definition, the function vq is essentially supported in a square of radius p . Hence we 
approximately have £i — r C {£ : |£ 2 | < P" 1 }- Assuming this condition, if 9 ^ 7r/2, then, for 
sufficiently small a, 

W(a|6 - r|)y((a;(6 - r) - 0)A/S) = for all r. 
Thus, we can conclude that 



|<(ir*F i ).t;g ) 7a,^>|<< 



sin(0) - 1 



-A' 



Now, continuing the computation in (10) by using a change of variables, we obtain 
((H^F^-vq,^) 

•^(I6l)e" i(fel/a)ei ^i- 
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Since W is compactly supported, the size of the support of the function G a , defined by 
6.(6) = ^" 1/4 • I6r J v q (t)W(\^ - ar\)V((u(a-% - r) - 9) / ^)e^ drW [U) , 

is independent of a. Using repeated partial integration, we conclude that 

|G(6)| < UGalU • dh/air" < a"' 1 /* ■ (\h/a\)- N . 

The result is then completed by invoking (9) and (11). □ 
Let us now introduce the cluster of significant curvelet coefficients for Cj. First, we define 
clusters for the manipulated pieces Hj^q, which are then 'lifted' to a cluster for Cj. Intuitively, 
the cluster around Hj^Q should spatially contain a carefully growing neighborhood of the 
discontinuity {0} x [— p, p] and directionally contain again a carefully growing neighborhood 
now of the direction of the discontinuity. For this, measuring the spatial distances between 
a point x and a set A with 

d 2 (x, A) = min \\x - a\\ 2 , x G M 2 , A C M 2 , 

agA 

we define the neighborhood of the discontinuity {0} x [— p, p] in phase space by 

A^ 5 (%) = {b£R 2 : d 2 (b, {0} x [-2p, 2p]) < a)~ £ } x [0, y/afi, 

where s G (0,|). We are then led to define its corresponding neighborhood for discrete 
parameter sets - setting 9jj = ir£/2^ 2 and bj^,t = Re je D 2 -]k - by 

S j = {U,k,l):(b j M,e u )eArf a (a j )}. 

In order to 'lift' these clusters to a cluster for Cj, we will make use of the filtering matrix 
associated with the filter Fj, i.e., of the matrix M F . = ((•y ri ,Fj *7^'))^y- We next need to 
recall that the action of a diffeomorphism 0q on a distribution / by <pqf = f o <pQ induces a 
linear transformation on the space of curvelet coefficients. With a(f) the curvelet coefficients 
of / and 0(f) the curvelet coefficients of <pqf , we obtain a linear operator defined by 

M 4Q (a(f)) = f3(f). 

Letting now t n , n % denote the amplitude of the n 2 'th largest element of the r/'th column of 
the matrix Mp. ■ M^ Q yi, we can define the overall cluster set for Cj by 

Q 

where 

s i,j,Q = ■ rf e Sj and \M F . ■ M {4>Q) -i(rj,rj')\ > t^,^}. 

The following result will be needed for estimating the relative sparsity of each Cj with 
respect to the just defined cluster Sij. 

Lemma 2.2. Let atj = ((Hj t Q, 7^))^ denote the curvelet frame coefficients of Hj Q = (H * 
Fj) ■ vq. Then for all sufficiently large j , 
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Proof. This proof follows the lines of the proof of [19, Lem. 6.2] very closely, wherefore we 
decided not to state it explicitly. It should just be mentioned that here we use Lemma 2.1 
instead of [19, Lem. 6.1], which leads to a difference in the asymptotic behavior by 2~ J . □ 
The decay estimate of the cluster approximate error 8\j is then given by the following 
result. 



Lemma 2.3. 

8ij= E l<7„C i )|=o(2-^), j^oc. 
veA\Sx,j 

Proof. Since the proof is very similar to [19, Lem. 8.5], we will not state it, but merely 
mention that instead of [19, Lem. 6.2] we here apply Lemma 2.2. □ 



2.2. Texture. The decay estimate of the cluster approximation error 52j is given by the 
following result. 

Lemma 2.4. If there exist rij, r%j > such that 

J2 = o(2~ j/2 ), j oo, (12) 

(m,n)^B(0,r 1 j)xB(0,fj i j) m€Z 2 
n£Aj 

then 

Szj = o(2~i/ 2 ), j ->> oo. 

Proof. First, 

$2,3 = E \((9 Sj )m,n,Tj)\. 

(m,n)0B(O,ri,j))xB(O,r 2 ,3 

Similar to the proof of Lemma 1.2, we have 

r — |m — m| 

j -»■ oo. 



\{(g Sj )m,n,Tj)\ ~ l^-(ri) ■ | E d ™> n 

m£Z 2 



m — m| 

e 2 



This immediately implies the claim. □ 
To derive a better understanding, we analyze again the situation examined in Example 
1.1. 

Example 2.1. We here consider the situation that 

\d m , n \ ~ \m\-^ ■ \n\-V +5 \ 5 > 0, 
We aim to derive conditions on rij,r2,j > such that (12) is satisfied, i.e., such that 

T 1 -T 2 + T 3 -T 4 = o(2'^ 2 ), j^oo, (13) 
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where 



Firstly, 



hence 



Secondly, 



hence 

Thirdly, 
And, fourthly, 



hence 



Tl 

T 4 



E 


[° 


M ^ — -v , ., , Co i jf\ |m-m| " 

e 2 + ^ m | i i+d ) e 2 


\m\>nj 




m£Z 2 \{0} 


E 


\n\ 


-(2+5) 


nGZ 2 n»4j 






E 




e 2 + ^ \ m \ ^+ b > e 2 


\m\<rij 




m£ff\{0} 



E 

nGZ 2 n^jnB(0,r2,j) c 



n 



-(2+5) 



C • / e 2 . ro ] r 



ri, 



|y| 



-(2+5) 



e 2 r 



-(2+5) . rdr 



To ~ I — 



i,i> 



2' 



7a ~ 1. 



2J + 1 
s :7 



max — ,r 2j 



-(2+5) . rdr 



: r 2j =w(f 

-5 

: r 2 ,=0 



: r aj = W [3i 



Applying (14) -(17) to (13) yields 
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Taking the energy matching condition (7) from Example 1.1 into account, 



: r 2 ,j 

5 ltj ~ r$ ■ + { 

2 2+2* 3 : r 2 j 



■ 2-? 
OJ ' 

' 'J 



However, 2 2 + 25 *' = cj(2 ^ 2 ). Hence, condition (12) only fulfilled, if and only if 



OJ 



-) 



1/6 



u (2 2 *(W ) and r 2 ,j=LO 



2-' 



oj ( 2 2 < 1 +«) j ) . 



3. Cluster Coherence 



3.1. Interaction of Gabor Elements and Curvelets. We start by estimating an inner 
product of a Gabor element and a curvelet. Depending on how Sj relates to 2\ we have to 
distinguish three cases. Note however, that the decay rate in all three cases is 0(2~ J//4 ) as 
j -> oo. 

Lemma 3.1. For each N = 1,2,..., there is a constant cn so that, 

(i) if Sj = o{2?l 2 ) as j — > oo, 

|<7a„ M , (9s 3 )m,n)\ < C N • 2"^ 4 • Sj ■ 1 \/,. ,,..„ ( // ) • (\ Sj b - f k)^, 

(ii) z/sj = f2(2 J / 2 ) and s^- = o{2 3 ) as j — > oo, 



\{la jA6 , (g Sj ) m ,n)\ < C N ■ 2-1/* ■ 1 A , 9 , s » ' <K S A ~ ^ ^2 



lll-> 



))k> 



-N 



(ii) if Sj = f2(2 J / 2 ) and Sj = o(2 J ) as j — > oo, 

\(la hb fi, (g Sj ) m ,n)\ < C N ■ 2 3 ^ 4 ■ Sf ■ l Ma . e , Sj {n) ■ d b - §-\D 2J Re)- N , 

where M aj ^ iSj is defined as in Theorem 1.1. 

Figure 5 illustrates the set M a . t g tS . for 6 = 0, whereas the three cases in which Lemma 3.1 
is split into are illustrated in Figure 6. 



2J/ 2 + l 



SUPP 7a, ,0,0 




2 j + 1 +l 



Figure 5. Support of the set M aj:0 , Sj . 
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(b) 

Figure 6. Relation between the support of a curvelet 7j,o,fc an d the support 
of a Gabor element (g aj )m,(m,o)- ( a ) : Case Sj = o(2 J / 2 ). (b): Case Sj = Q(2^ 2 ) 
and sj = o(2 j ). (c): Case Sj = tt(2 j ). 



Proof of Lemma 3.1. First, by Plancherel and the definition of M aji e tS ., 

(la h b,6, (g S )m,n) 



c-af I W( aj \C\)V((u;-9)/^)g s ^ 



3/4 _i 

It now remains to estimate 
T 



' " V l M M {n) I W( aj \t\)V{(u> - 9)/y^)g(j: - n)/ 6 2 ^d£. (18) 



W(a^\)V((oo - 9)/^)g(j: - n)e n " 



in the three cases the claim is split into. For this, in all three cases, WLOG we assume that 
6 = 0. 

Case Sj = o{2^ 2 ) as j — > oo. With C — i an d letting oo(-) denote the angular component, 



in 



W(ajSj\t\)V(ou(sjO/V^)9(( ~ n)e^ b -^d( 



Applying integration by parts, for any k = 1,2, 



\T\ = s 2 j-\ S jb-f\ 



< s<-\s 3 b-f 



m I — k 



A k [W(a, Sj \C\)V(u(s,0/^a-)g(C - n)]e^ b -^d( 
|A fc [lF(a jSj |C|)V(u;( Sj C)/v / %)^(C " n)]\d(. 



Hence 



l + | s . 6 _s|fc).| T | < J \W(ajSj\C\)V(u;( S jO/^)g(C-nM 

+ J \A k [W(ajSj\(\)V(uj(sj()/^a-yg((-n)}\d(. 
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It can be shown that, for each k, there exists < oo such that, for all aj, 

,fe 



iWia^lCDViuis.O/^MC-nM + l ^[W^s^V^s^Q / ^)g(t-n)}\dt < c k . 
Further, for each k = 1, 2, 



k 



(\ Sj b - f \r = a + \ Sj b - f < -(i + | fli 6 - f n, 

we obtain 

m < Cw - Sj 2 -(i^-fi)- iv . 

Reinserting 6 1 and using (18), claim (i) is proved. 

Case sj = tt(2i/ 2 ) and Sj = o(2-») as j oo. With ( = (sj 1 ^, 2" J ' /2 6) =: </?j(0, 



ITI 



2^ 2 . Sj / ^(a i |^7 1 (C)|)^(^7 1 (C))/^(Ci-ni,2 i/ VC2-^) 



Repeated integration by parts - similar as in the first case - implies 

\t\ < c N ■ yi* ■ Sj ■ (Ksjh - ?f,2^ 2 (b 2 - ^))\r N . 

Reinserting 9 and using (18), proves claim (ii). 
Case Sj = Q(2 j ) as j ->• oo. With ( = D 2 -j£, 



ITI 



2 3J/2 / W{\C\)V{u{D 2i C)/ s /a-)g{D 2 ^-n) 



Repeated integration by parts - similar as in the first case - implies 



|T|<^.2 3 ^.(| 6 -^| D2j .)- 



■N 



Claim (iii) now follows by reinserting 9 and using (18). 

3.2. Cluster of Curvelets. We can now prove negligible cluster coherence of S\j. 
Lemma 3.2. 

(aOu = Ai c (Sij, {{g S] )\}] {7r,}) -> 0, as j oo. 
Proof. First, we observe that the definition of Sij implies 

(A*c)lj = max Y] 1(7^,6,(9, (g Bj )m,n)\ 
m,n 

= maxV V \('ya j ,b,9,(98 J )m,n)\ 

m,n z — ' *— ' 

< #Q-maxmax ) Kla^b.e, {g Sj )m,n)\ 

Q m,n • ' 

(b,0)es lijtQ 



□ 
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Let us now first prove the claim in case that <f)Q = Id, i.e., Mq = Id, hence <Sij,q = Sj. 
Notice that WLOG we can assume that m = and n = (2 J , 0)(g M aj} g )Sj ). Hence it remains 
to continue the estimate 

(/i c )i,j < #Q • max Y 1(7^,6,0, (0^)0,(2* ,o)) I- (19) 
{b,e)es ltjtQ 

We now split the proof - similar to the splitting in Lemma 3.1 - in three cases. 
Case Sj = o(2 J / 2 ) as j — > 00. By (19) and Lemma 3.1, 

(/i c )ij < c N -2-^l 4 - S] Y (1^(2-^1,2-^)1)-^ 

< CiV • 2" 3 ^ 4 • Si • vr- 1 • 2^ ]T (\sfl-^k 2 \y N 

|fc 2 |<2p+2-i( 1 /2- E ) 

< CiV • 2~ 3 ^ ■ s 3 ■ 2 2£ > ■ sj 1 • (2p + 2-^/ 2 ^) 

Thus, since e < ~, 

(aOij -»■ 0, as j 00, 

which settles this case. 

Case Sj = Q(2^ 2 ) and Sj = o{2^) as j 00. By (19) and Lemma 3.1, 

< cn-2-^ Y (\(sjbi,2 j / 2 b 2 )\ Re )- N 

< c N -2-^.^. Y E (1(^2-^1,^)1)^ 

|fci|<2 £ J |fc 2 |<2p+2-3(i/2-E) 

< c N -2-j/ 4 -2 2 ^ Yl (N>~* 

|fc 2 |<2p+2-J'( 1 /2-e) 

Since the last sum is bounded by a constant, 

< c N ■ 2^ 2£ -s). 

Since £ < |, 

(/i c )ij -> 0, asj->oo, 

which proves the claim for this case. 

Case Sj = f2(2 J ) as j — > 00. By (19) and Lemma 3.1, 

GUcK; < C N -2^. S f Y (\b\D 2J R e r N 

(k,i)eSj 

< c N .2^.sj'.n-i.Y(\k\r N 

k 

< c N -2^ 4 -s-\ 
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Hence, 

(^c)ij ->■ 0, as j ->■ oo. 

Hence also this case is proved. 

The general claim now follows by using similar arguments as in [19, Lem. 8.8]. □ 

3.3. Cluster of Gabor Elements. Finally, we arrive at the study of the cluster coherence 
of S 2 j. 

Lemma 3.3. Suppose that one of conditions (i) and (ii) is satisfied: 

(i) Suppose Sj = o(2 J ) as j — > oo, and r 2 j is chosen such that 

\B(0, r 2j ) n M a . j0>s . f]Z 2 \ = o(2 3 ^ 4 ■ sj 1 ), as j oo. 

(ii) Suppose Sj = f2(2 J ) as j — > oo, and r±j is chosen such that 

\B(0,r 1}j ) n Z 2 | = o(2" 3j/4 • Sj ), as j -> oo. 

Then 

(j*chj = A*c(52j, {i v }; {(g Sj )x}) -»■ o, as j -»■ oo. 

Proof. First, 



(/i c ) 2J = max |(7 % A^(^)m,n)| 



(m,n)£<S 2 ,j 

XI 1(7^,0,0, (^)m,n) |, (20) 
(m,n)e52,j 

since, for symmetry reasons, WLOG we can assume that the maximum is attained in b = 
and 9 = 0. The two cases will now be dealt with separately. We start with the first, which 
- similar to the splitting in Lemma 3.1 - we separate into two subcases. 
Case Sj = o(2^ 2 ) as j — > oo. By (20) and Lemma 3.1, 

(t'.h, < rx ■ 2 > 1 • Sj (\T\)~ N E L 

Since the first sum is bounded by a constant and by condition (i), 

(fi>c)"2,j = o(l), as j -> oo, 

this case is settled. 

Case Sj = tt(2^ 2 ) and Sj = o{2^) as j — ?• oo. By (20) and Lemma 3.1, 

meS(0,n j ) neB(0,r 2 ,j )nM aj >0 , Sj nz 2 

Since 

PT\ pi? ^ ^ S ^7*1 

' \2^ 2 sj 1 x\)- N dx = 2-i/ 2 -Sj J ' '\\x\)- N dx~2-i/ 2 - Sj , 
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we obtain - also exploiting condition (i), 

Ockj < c N ■ 2~ 3j/A ■ Sj ^2 1 = (!)> as j -too. 

This proves the claim for this case. 

Case Sj = f2(2 J ) as j — > oo. By (20) and Lemma 3.1, 

(„ c ) 2 , < c N ■ 2^ ■ s? £ (\b-§-\ D2J )- N E I- 

meB(o, riJ ) neB(0,r 2lj )nAf a , s ,nz 2 



Since \M a . ; o !S . HZ | < const, 

-N 



meB(p,ri,j) 

< c N ■ 2 3j/A ■ sj l \{m E Z 2 : |m| < rij}|. 

By condition (ii), 

Oeki = o(l), asj^oo, 
The lemma is proved. □ 

4. Proofs 
4.1. Proofs of Results from Section 1. 

4.1.1. Proof of Lemma 1.1. Proof. We first observe that, for W sufficiently nice, 

WC.Wl = J |^.|£|)| 2 |C,(0| 2 ^ ~ J a m\ 2 d£, (21) 
where Aj = {£ : 2 j < |f | < 2 j+1 }. Using (3), we can continue (21) to 

\\ C j\\l ~ / r_2rfr ~ 2 " J - D 

4.1.2. Proof of Lemma 1.2. First, by the change of variable oj = £/s and the support condi- 
tion on g, 

WTsjWl = J2YlJ d m,nd^W 2 (a j \C\)g s ^ - sn)g s (£ - sn)e im ^^d^ 

m,n fh,h 

= s 2 ■ Y J ' d m ^ n dr h ^W 2 (a j s\uj\)g s (s(uj - n))g 8 (s(u - fi))e lV ^^duj 



m,n,m,n 
\n-h\<l 



^ d m , n dfh,h J W 2 {a j s\u\)g{u - n)g(u - n)e im ^ w du. (22) 



m,n,m,n 

\n-h\<l 
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For 'sufficiently nice' W, 

IV (ajs|a;|)p(a; - ra)g(u; - n)e dtu ~ / g(u — n)g(u — n)e ~5~ aw 



hence, continuing (22), and taking into account that for each n, there exist a finite number 
of n's independent on j satisfying \n — n\ < 1, 

Il^jll2 ~ d m,ndm,h / g(u -n)g(u -h)e lT0 -^ u duj 

m,n,m,n a iJ 

|ra-n|<l 

^ ^ d m , n drh,h / #(w - n)^(a; - n^^V^da; 



m,m „,fi e z2 n _4 s 
In— nj<l 



^ d m ,ndrn, n f \g(uj - n)^^" dw . (23) 



m,mna 2 nAj 



Now, WLOG we assume that the support of g is always entirely contained in A s f then, by 
(23), 

W^Al ~ J2 d m,nd^n j ~ fl)\ 2 ' du . (24) 

Since \g(x) \ ~ e - ^', 

|^(w-n)|V-3- "do; = e J — n - / |#(w)|V— 

= e « ffl i fi ».^* y *)(2S=*) 

|m — m | 

~ e 2 . 

Thus, by (24), 



| m — m| 
2 

) 



m,rh n£Z 2 nA s ,j 

and the lemma is proved. □ 



4.1.3. Proof of Theorem 1.1. The theorem follows immediately from applying Lemmata 2.4, 
2.3, 3.3, and 3.2 to Proposition 1.1. □ 



4.1.4. Proof of Corollary 1.1. We choose r± j > such that 

nj = (^(2^), as j -»■ oo, 

and r 2j - > such that 

r 2j - = u(2^+^ j ), as j ->■ oo. (25) 
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It now suffices to check the sufficient conditions posed in Theorem 1.1. Firstly, by Example 
2.1, 

E x — \rh — m\ ■ tn 

\dm,n\e ~ = o(2" J/i ), j oo. 

(m,n)?B(0,r lj )xfl(0,r 2 j) mgZ 2 

Now notice that the energy matching condition in this case, i.e., (7), implies that Sj = Q(2^ 2 ) 
and Sj = o(2 J ) as j — > oo. Hence we need to prove 

|B(0, r 2J ) n M a . AtJ HZ 2 \ = o(2 3 ^ 4 ■ sj 1 ) = o(2^V), as j -> oo. (26) 
For this, by (25), 

\B(0, r 2J ) n M o . j0 , s . n Z 2 | = |M o . j0 , s . n Z 2 | ~ s" 1 • 2^ • sj 1 ■ V' 2 = sj 2 ■ 2 3 ^ 2 , as j -> oo. 
Since, by (7), 

claim (26) is satisfied if and only if 5 > 1, which was assumed. 

The claim now follows from Theorem 1.1. □ 
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